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Prior and Posterior 


9.1.1 Prior and Posterior 

Let X be the random variable whose value we try to estimate. Let Y be the observed random 
variable. That is, we have observed Y — y, and we would like to estimate X. Assuming 
both X and Y are discrete, we can write 


P(X = x\Y = y) 


P(X — x,Y — y) 

P(Y = y) 

P(Y = y\X = x)P(X = x ) 

W = y) 


Using our notation for PMF and conditional PMF, the above equation can be rewritten as 


Px\y( x \v) 


Py\x(v\x)Px{x) 

PAy) 


The above equation, as we have seen before, is just one way of writing Bayes' rule. If either 
X or Y are continuous random variables, we can replace the corresponding PMF with PDF in 
the above formula. For example, if X is a continuous random variable, while Y is discrete we 
can write 


Py\x(y\x)fx(x) 

AM ' 

To find the denominator (Py{y) or fyil/)), we often use the law of total probability. Let's 
look at an example. 


fx\Y( x \y) 


Example 9.3 

Let X ~ Uniform(0, 1). Suppose that we know 

Y | X — x ~ Geometric(x). 
Find the posterior density of X given Y — 2, fx\y{ x 12). 

• Solution 

o Using Bayes' rule we have 
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/xiy( aj |2) — 


Py\x(?\ x ) jx( x ) 


M 2) 

We know y | X = a: ~ Geometric(x) , so 

^V|jr(2/|») = *(1 for y — 1,2, 

Therefore, 

^V|x(2|z) = *(1 -*). 

To find Py(2) , we can use the law of total probability 

/ oo 

i V|x( 2 l*)/x(») da; 

•oo 

= f x(l — x) • 

Jo 

1 

~ 6 


1 da: 


Therefore, we obtain 


/x'|y( aj |2) — 


a:(l — a:) • 1 


= 6a:(l — a:), for 0 < x < 1 . 


For the remainder of this chapter, for simplicity, we often write the posterior PDF as 


fx\y(x\y) 


fy\x(y\x)fx(x) 

fr(y) 


which implies that both X and Y are continuous. Nevertheless, we understand that if either 
X or Y is discrete, we need to replace the PDF by the corresponding PMF. 
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